Sentiment/Textual Analysis

Correlogram

First, I change all factor variables (anything with response values that can be considered discrete) to values between -2 and 2, then I correlate all variables in a correlogram which visualizes the cross correlations of each square. The results are pretty much what one would expect, high scores in one area are associated with high scores in another area. The only thing to really notice here is that in general people seem to have a lower likelihood to associate the belief that they will apply what they learned in the next 4 to 6 weeks with most other variables, although it is not that much lower. This could be a lower association in that teachers believe they will be more likely to apply their learning when other variables have lower scores, or the inverse, which seems vastly more likely, especially since most scores are very high on average.

To supplement the results of the correlogram I create a summary statistics table where you can see the number and percentage of each level of agreeing:

Likeliness to Agree/Disagree
Group and Subcategories N = 7,9271
Learning Session Satisfaction
Strongly disagree 53 (0.7%)
Disagree 82 (1.1%)
Neither agree nor disagree 442 (6.0%)
Agree 1,687 (23%)
Strongly agree 5,057 (69%)
Unknown 606
Todays Topic Was Relevant For My Role
Strongly disagree 45 (0.7%)
Disagree 79 (1.2%)
Neither agree nor disagree 356 (5.4%)
Agree 1,301 (20%)
Strongly agree 4,832 (73%)
Unknown 1,314
Designed To Help Me Learn
Strongly disagree 49 (0.7%)
Disagree 104 (1.5%)
Neither agree nor disagree 451 (6.7%)
Agree 1,371 (20%)
Strongly agree 4,757 (71%)
Unknown 1,195
Likely To Apply 46 Weeks
Strongly disagree 41 (0.6%)
Disagree 66 (1.0%)
Neither agree nor disagree 383 (5.8%)
Agree 1,254 (19%)
Strongly agree 4,887 (74%)
Unknown 1,296
S/he Facilitated The Content Clearly 12
Strongly disagree 11 (0.1%)
Disagree 25 (0.3%)
Neither agree nor disagree 196 (2.7%)
Agree 1,140 (16%)
Strongly agree 5,977 (81%)
Unknown 578
S/he Facilitated The Content Clearly 16
Strongly disagree 2 (0.2%)
Disagree 8 (0.9%)
Neither agree nor disagree 62 (6.8%)
Agree 216 (24%)
Strongly agree 620 (68%)
Unknown 7,019
S/he Effectively Built A Community Of Learners 17
Strongly disagree 4 (0.4%)
Disagree 8 (0.9%)
Neither agree nor disagree 67 (7.4%)
Agree 209 (23%)
Strongly agree 620 (68%)
Unknown 7,019
S/he Effectively Built A Community Of Learners 13
Strongly disagree 11 (0.1%)
Disagree 40 (0.5%)
Neither agree nor disagree 231 (3.1%)
Agree 1,098 (15%)
Strongly agree 5,965 (81%)
Unknown 582
Recommend To A Colleague Or Friend 10.00 (8.00, 10.00)
Unknown 406

1 Statistics presented: n (%); Median (IQR)

Textual Data

I look at the textual data first by simply counting words and seeing what appears most frequently, the first occurrence is NAs which I filter out, along with many other ways to say “no response”. After converting the text columns to one word format and filter out stop words, I group by each kind of text column below.

The most common words for activities that support learning appear to be planning, discussion, and student by a wide margin. This seems pretty reasonable and there isn’t much to read into in that regard.

Initially its obvious that most people don’t bother to fill out the additional comments part, but the second word count chart here for additional comments indicates there might be some timeliness issues or other need for better partitioning.

Again it looks like time is a big factor, lessons and planning show up here a lot too.

A lot of expectable themes show up looking at things to try out, teachers seem to be very interested in “text”, “learning strategy”, and relating it to “students.”

“Collaboration” appears to be working well, and “discussions” appears too so group based stuff seems very successful. “Time” makes another appearance as well.

The most common words for people who bothered to fill out their reason for the rating was that it was “informative”, “helpful”, and “knowledgeable” which are all positive. “Time” and “feel” make appearances at the bottom indicating some potential negative feedback.

Looks like people are doing a pretty good job in general.

Bigram

Let’s take a look all the responses at once, first as words, then as a bigram, we see what we have been seeing this whole time, more time is a big request:

Of course we have to look at the actual data to interpret what more time means - here I quickly sample 30 random comments from the 232 unique entries that have “more time” in them to see what most say. There’s no specific pattern that emerges other than a general request for more time which almost entirely comes under the “What could have improved your experience” question column.

Responses Involving More Time
Response Text response_type date_for_the_session professional_training_session select_your_site_district_parish_or_network select_the_best_description_for_your_role select_the_grade_band_s_you_focused_on
38 The whole program is wonderful, but I'm most excited to have the lessons ready so that I can spend more time on great teaching strategies and differentiation. learning_to_try_out 2020-05-19 EL Virtual Bootcamp ESU 2, Nebraska Teacher or specialist Grades K-2, Grades 3-5, Grades 6-8
42 Having more time to plan what_could_have_improved_your_experience 2020-01-13 General: ELA Delaware DOE School-based coach Grades 6-8, Grades 9-12
65 always could use more time what_could_have_improved_your_experience 2019-07-10 Guidebooks CM-1 Iberia Parish Teacher or specialist NA
215 more time with ed reports what_could_have_improved_your_experience 2019-10-28 Innovation Network Fellowship, Meeting 1 Nebraska Innovation Network Fellowship Teacher or specialist NA
118 It was cool. If there was a little more time to work on our assignments that would be great. what_could_have_improved_your_experience 2019-09-25 Guidebooks CM-4 Louisiana State Content Leader Training District-based coach/curriculum and instruction specialist NA
178 Maybe more time what_could_have_improved_your_experience 2019-09-13 Guidebooks Writing PLC Washington Parish Teacher or specialist NA
11 Spending more time digging into the module. what_could_have_improved_your_experience 2019-08-27 EL K-5 BootCamp, Day 2 Baychester Academy Teacher or specialist NA
3 Having more time for PDs what_could_have_improved_your_experience 2020-02-03 EL K-5 BootCamp, Day 1 NYC District 11, PS 121 Teacher or specialist Grades 3-5
128 A little more time for planning what_could_have_improved_your_experience 2019-10-14 Guidebooks CM-5 Allen Parish Teacher or specialist NA
170 more time what_could_have_improved_your_experience 2019-11-14 Guidebooks Writing PLC Washington Parish Teacher or specialist NA
198 Great, but still need more time to plan. why_did_you_choose_this_rating 2020-01-06 IM Making Math Visible, Day 1 Springfield Empowerment Zone Teacher or specialist Grades 6-8
68 More time factored in for debate and to grapple with ideas. I felt like I wanted to spend more time talking about what Zach said when he challenged the strategies idea. I think to really be a community of learners we need some time to do the heavy lifting and really work out the validity of these ideas among ourselves by challenging them and defending them. what_could_have_improved_your_experience 2019-08-13 Guidebooks CM-1 Digital Pioneers Academy Teacher or specialist NA
58 Spending more time unpacking each unit but I get #Trusttheprocess what_could_have_improved_your_experience 2019-06-13 Guidebooks CM-0 Lafayette Parish Teacher or specialist NA
50 Please provide more time to reflect or discuss (2 minutes of quiet think time +) what_could_have_improved_your_experience 2020-01-13 General: Math Delaware DOE School-based coach Grades 6-8
232 A little more time to complete the activities. what_could_have_improved_your_experience NA NA Robinhood Schools (NYC) Teacher or specialist Grades K-2
30 Given more time to collaborate with coaches to design an action plan what_could_have_improved_your_experience 2019-11-18 EL K-5 Speaking & Listening, Day 2 Linden Tree School Teacher or specialist NA
133 Although the time for planning was adequate, we could always use even more time to plan. what_could_have_improved_your_experience 2019-12-02 Guidebooks CM-5 Lafayette Parish Teacher or specialist NA
211 diving more into the content I am using- more time planning what_could_have_improved_your_experience 2019-09-27 IM Student Thinking, Day 1 Freire Charter Schools Teacher or specialist NA
204 Needed more time on specific activities. do_you_have_additional_comments 2019-10-14 IM Student Thinking, Day 1 Building 21 Teacher or specialist NA
75 The timing constraints were a bit much. Take more time on specifics do_you_have_additional_comments 2019-06-20 Guidebooks CM-1 Lafayette Parish Teacher or specialist NA
225 Spending more time with the scripting and not having to rush through the different scenarios would have been even better. Time could have been taken from the change management module which could have been covered more quickly. what_could_have_improved_your_experience 2020-01-23 School Leaders: GuideBooks Leaders Workshop Calcasieu Parish School-based administrator or supervisor Grades 6-8
229 We needed more time. We never were able to observe and practice / norm / align our feedback. what_could_have_improved_your_experience 2019-10-28 School Leaders: IM SL-1 Freire Charter Schools School-based administrator or supervisor NA
19 Just more time... what_could_have_improved_your_experience 2020-02-24 EL K-5 Close Reading & Complex Texts, Day 1 NYC District 11, PS 41 Teacher or specialist Grades 3-5
169 Spending more time on the afternoon activity rather than essays from the morning what_could_have_improved_your_experience 2020-01-24 Guidebooks Writing PLC Washington Parish Teacher or specialist Grades 9-12
189 having more time to look at the curriculum and ask questions as I went through what_could_have_improved_your_experience 2019-08-06 IM Bootcamp, Day 2: Freire Freire Charter Schools Teacher or specialist NA
132 more time to plan and collaborate with peers what_could_have_improved_your_experience 2019-12-02 Guidebooks CM-5 Lafayette Parish Teacher or specialist NA
88 more time out of seats what_could_have_improved_your_experience 2019-07-24 Guidebooks CM-2 East Baton Rouge Parish Teacher or specialist NA
115 more time what_could_have_improved_your_experience 2019-11-05 Guidebooks CM-4 Iberia Parish Teacher or specialist NA
18 more time to actually prepare for a lesson I will teach in the near future. ( 1-3 days from today. what_could_have_improved_your_experience 2020-02-24 EL K-5 Close Reading & Complex Texts, Day 1 NYC District 11, PS 41 Teacher or specialist Grades 3-5
217 More likely than not, but also need more time with this team....no why_did_you_choose_this_rating 2019-10-28 Innovation Network Fellowship, Meeting 1 Nebraska Innovation Network Fellowship District/network/state-level professional or administrator NA

Networking

Now I create a network of the bigrams, I notice there are some weird special characters in the data: ö, ä, ñ appear on the center-right of the Fruchterman-Reingold network, as well as in the upper right. I took a look at the appearances in the data and it seems like some stuff got miscoded on entry in the survey, there’s 655 entries that contain ä, and a lot of them look like this: “It‚Äö√Ñ√¥s boring”, which is pretty difficult to interpret. The network itself establishes an “alpha” (the level of opaqueness) based on how rare or common the bigram is, and directionality is established by the arrows. Intepreting this is more of an art than a science, but my main takeaway here is that working in small groups is very highly valued based on the central node in the network where many different words appear to connect to “group”.

Sentiment Analysis with AFINN

I use the AFINN lexicon for sentiment analysis, which gives a numeric sentiment value for each word, with positive or negative numbers indicating the direction of the sentiment.

I visualize this with a plot that shows words going in the “right” or “wrong” direction based on the sentiment analysis using a bar plot. A lot of the “good” ones here are also not great when taken into context of being preceded by “not”. I filter for the top 20 values of contribution which is the count multiplied by the “value” according to AFINN in this chart:

With these word combinations it is easy to comb through the data and see where negative comments are made. Below I again print out the relevant text containing “not + word” to demonstrate actual responses I am aggregating:

Potentially Negative Responses
Response Text response_type date_for_the_session professional_training_session select_your_site_district_parish_or_network select_the_best_description_for_your_role select_the_grade_band_s_you_focused_on
1 To not worry so much about teaching the curriculum with fidelity but rather skillfully and with integrity. learning_to_try_out 2019-10-16 EL K-5 BootCamp, Day 1 Massachusetts DESE NA NA
2 I think it was a great program with a lot of great learning built in. It flowed nicely and did not allow for too much lag time. I think the last part felt rushed where could have used a bit more time for planning. why_did_you_choose_this_rating 2019-10-03 EL K-5 Speaking & Listening, Day 1 Linden Tree School Teacher or specialist NA
3 Learning that I am not alone in my struggle. overall_what_went_well_in_this_professional_learning 2020-01-27 EL K-5 Speaking & Listening, Day 2 NYC District 11, PS 41 Teacher or specialist Grades 3-5
4 The discussions by the instructors were very inspiring because they were genuinely interested in us and dedicated to helping us see the philosophy of this curriculum. However my answer to the question of recommending this course is low because of the covid situation in our world. My answer is not because of the integrity of the course, the suggestions or the "gist" of the course. The instructors were excellent. The philosophy of equity and responsibility is also. But I cannot recommend this course in light of covid-19. I feel teaching EL through distance learning with integrity would not be possible. which_activities_best_supported_your_learning 2020-05-19 EL Virtual Bootcamp ESU 2, Nebraska Teacher or specialist Grades 3-5
5 It was not easy to learn this virtually why_did_you_choose_this_rating 2020-05-19 EL Virtual Bootcamp ESU 2, Nebraska Teacher or specialist Grades 3-5
6 Very well presented. Time not wasted. why_did_you_choose_this_rating 2020-01-13 General: Math Delaware DOE Teacher or specialist Grades 9-12
7 What to do when you have people who do not want to participate. overall_what_went_well_in_this_professional_learning 2019-11-07 Guidebooks CLM-8 Louisiana State Content Leader Training Classroom teacher NA
8 The program was not intimidating. why_did_you_choose_this_rating 2020-01-15 Guidebooks CLM-9 East Baton Rouge Parish ELA Content Leader Grades 9-12
9 The program was not intimidating. why_did_you_choose_this_rating 2020-01-15 Guidebooks CLM-9 East Baton Rouge Parish ELA Content Leader Grades 9-12
10 Teaching main idea is not necessary and do not leave any thing out when teaching a lesson. learning_to_try_out 2019-06-12 Guidebooks CM-0 Lafayette Parish Teacher or specialist NA
11 It was not helpful to an experienced teacher. why_did_you_choose_this_rating 2019-08-22 Guidebooks CM-0 Allen Parish Teacher or specialist NA
12 That I was not alone in my feelings. overall_what_went_well_in_this_professional_learning 2019-08-06 Guidebooks CM-0 Calcasieu Parish Teacher or specialist NA
13 I was not feeling well. I enjoyed the PD. what_could_have_improved_your_experience 2019-07-22 Guidebooks CM-0 Lafayette Parish Teacher or specialist NA
14 The vegetarian lunch was not good at all. what_could_have_improved_your_experience 2019-06-24 Guidebooks CM-0 Louisiana State Content Leader Training District-based coach/curriculum and instruction specialist NA
15 Room set up was not great do_you_have_additional_comments 2019-06-24 Guidebooks CM-0 Louisiana State Content Leader Training School-based coach NA
16 I needed to move around a little more. And I did not like that we had to switch seats when we arrived. I wanted my seat from yesterday, and my neighbor! what_could_have_improved_your_experience 2019-06-12 Guidebooks CM-0 Lafayette Parish Teacher or specialist NA
17 Teaching students skills THROUGH text and not isolated skills. This stuck out to me because I feel as though a lot of lower elementary teaching was skills based, and this is a major shift in thinking. learning_to_try_out 2019-06-24 Guidebooks CM-0 Louisiana State Content Leader Training Classroom teacher NA
18 Chars with backs, stools are not comfortable. what_could_have_improved_your_experience 2019-08-07 Guidebooks CM-0 Lafayette Parish Teacher or specialist NA
19 Convention center is not responsive to its clients what_could_have_improved_your_experience 2019-06-25 Guidebooks CM-1 Louisiana State Content Leader Training Classroom teacher NA
20 The convention center needs to make sure the lunches are fresh, and they need to not interrupt sessions (e.g., bringing in chairs when our facilitator was presenting). what_could_have_improved_your_experience 2019-06-25 Guidebooks CM-1 Louisiana State Content Leader Training District-based coach/curriculum and instruction specialist NA
21 Most educators do not enjoy long, multi-days workshops or PDs. why_did_you_choose_this_rating 2019-07-23 Guidebooks CM-1 East Baton Rouge Parish Teacher or specialist NA
22 I did not like how we only had pastries for breakfast. Please consider having healthier options such as protein and fruits. do_you_have_additional_comments 2019-07-31 Guidebooks CM-2 Louisiana State Content Leader Training District-based coach/curriculum and instruction specialist NA
23 Lack of examples for high school. High school teachers should be grouped together. I teach 11th and 12th grade and spending 30 minutes on a 3rd grade text was not helpful to me. It would be helpful to have a conversation with others about high school curriculum and how all of this applies to US. what_could_have_improved_your_experience 2019-07-31 Guidebooks CM-2 Louisiana State Content Leader Training Classroom teacher NA
24 Thank you Sable for keeping laser-like focus despite the constant negative comments and sarcastic questions from one teacher in particular. You are a true professional. You did not allow negativity to persist and take over the sessions. do_you_have_additional_comments 2019-07-24 Guidebooks CM-2 Lafayette Parish Librarian NA
25 This is not focused towards teachers who are experienced with instructional shifts and design. why_did_you_choose_this_rating 2019-09-10 Guidebooks CM-3 Ascension Parish Teacher or specialist NA
26 Teaching Lab is doing a commendable job of supporting guidebooks, but I still feel that the Learnzillion Guidebook Curriculum is extremely lacking and does not effectively prepare students for what is actually assessed on the Leap2025. why_did_you_choose_this_rating 2019-10-08 Guidebooks CM-3 St. Martin Parish Teacher or specialist NA
27 Sometimes our facility is not best for what the instructor asks for such as emailing plans. Difficult to do if wifi does not cooperate. why_did_you_choose_this_rating 2019-10-25 Guidebooks CM-4 Lafayette Parish Teacher or specialist NA
28 The first presenter was wonderful but I feel that she moved way too quickly. I appreciate that we were let out early; however, if it is is at the risk of us not feeling like we could finish anything, I do not feel it is worth it. what_could_have_improved_your_experience 2019-10-25 Guidebooks CM-4 Lafayette Parish Teacher or specialist NA
29 I am glad to know that I am not alone when having trouble connecting relevant text to assertions. overall_what_went_well_in_this_professional_learning 2019-10-07 Guidebooks CM-4 Allen Parish Teacher or specialist NA
30 All that chocolate needs to be tossed. It's pretty melted and discolored. I appreciate you providing it, but it's not good anymore. what_could_have_improved_your_experience 2019-10-14 Guidebooks CM-5 Allen Parish Teacher or specialist NA
31 I did not leave with many tools to implement and ways to engage support my diverse learners. why_did_you_choose_this_rating 2019-09-19 Guidebooks Diverse Learners BootCamp, Day 1 Tangipahoa Parish Teacher or specialist NA
32 Discovered that I am not alone in my anxious feelings. overall_what_went_well_in_this_professional_learning 2019-08-28 Guidebooks Diverse Learners BootCamp, Day 1: Leaders Lafayette Parish School-based coach NA
33 The instructor (Jill) is great. The way she presented it was great. The content was not great. do_you_have_additional_comments 2019-09-17 Guidebooks Diverse Learners BootCamp, Day 2 Tangipahoa Parish Teacher or specialist NA
34 Very informative and not stingy on resources given. why_did_you_choose_this_rating 2019-08-29 Guidebooks Diverse Learners BootCamp, Day 2 Lafayette Parish School-based coach NA
35 Getting started earlier (not your fault and not mad about it). :) what_could_have_improved_your_experience 2019-12-12 Guidebooks Diverse Learners Fluency, Day 1 East Baton Rouge Parish Teacher or specialist Grades 3-5
36 It was very intimidating to have to teach in front of other teacher leaders. I am not a johnny on the spot planner. My planning is purposeful and we'll thought out. We barely had time to process our thinking. The activity was well intended however I think he could have been adjusted because of the audience. I do not want to return next time. what_could_have_improved_your_experience 2019-11-15 Guidebooks Diverse Learners Fluency, Day 1 Lafayette Parish School-based coach NA
37 I can't give an honest answer. I'm not excited about it. learning_to_try_out 2020-02-20 Guidebooks Diverse Learners Vocabulary, Day 2 Lafayette Parish School-based coach Grades 6-8
38 More time in breakout and although this is out of teaching labs hands, some teachers that were reluctant to this process haste shared: ‘at the beginning they did not share the same vision, growth mindset, buy in etc. however, they now see the benefit and are on board. It’s the lack of substitutes, the stress of not knowing if they will show, not knowing how to prepare and rushing to the location that is an overwhelming burden.’ Not all feel this way but I just wanted to share so that those concerned may understand people’s headspace. what_could_have_improved_your_experience 2020-03-04 Guidebooks Writing PLC Washington Parish Teacher or specialist Grades 9-12
39 Being able to collaborate with others and know that I am not alone overall_what_went_well_in_this_professional_learning 2019-10-08 Guidebooks Writing PLC Washington Parish Teacher or specialist NA
40 knowing i was not alone overall_what_went_well_in_this_professional_learning 2019-09-12 Guidebooks Writing PLC Washington Parish Teacher or specialist NA
41 I do not recommend anything until I see it in action over time. why_did_you_choose_this_rating 2019-08-22 IM Bootcamp, Day 2: B21 Building 21 Teacher or specialist NA
42 If I've already given my e-mail address I do not want the question to keep popping up. do_you_have_additional_comments 2019-08-22 IM Bootcamp, Day 2: B21 Building 21 Teacher or specialist NA
43 I am still not impressed by anything that has been presented to me. I have gained more from learning from my colleagues. why_did_you_choose_this_rating 2019-10-02 IM Student Thinking, Day 1 Cincinnati Public Schools/ STRIVE Teacher or specialist NA
44 Evan is fine and talking about teaching is always helpful, but I am not happy with being told how to think about my teaching. why_did_you_choose_this_rating 2019-12-11 IM Student Thinking, Day 2 Building 21 Teacher or specialist Grades 9-12
45 Better focus among our team - some were very distracted and not focused what_could_have_improved_your_experience 2019-10-29 Innovation Network Fellowship, Meeting 2 Nebraska Innovation Network Fellowship District/network-based coach or Curriculum and instruction specialist NA
46 A more structured way to share. People did not like having the microphone handed to them when not prepared. what_could_have_improved_your_experience 2019-10-29 Innovation Network Fellowship, Meeting 2 Nebraska Innovation Network Fellowship Professional Development NA
47 It was ok, but not outstanding. why_did_you_choose_this_rating 2019-10-29 Innovation Network Fellowship, Meeting 2 Nebraska Innovation Network Fellowship Teacher or specialist NA
48 This is an area that district leaders need to be focusing on and I do not think it naturally happens in the district- it is busy and decisions are not made this thoughtfully. why_did_you_choose_this_rating 2020-02-07 Innovation Network Fellowship, Meeting 4, Day 2 Nebraska Innovation Network Fellowship District/network-based coach or Curriculum and instruction specialist All grades K-12
49 It was better than average but not great- it appeared the team was rushed; too many adults in one room why_did_you_choose_this_rating 2019-11-13 School Leaders: Lafayette SL-3 Lafayette Parish School-based administrator or supervisor NA

Response 4 in the table jumps out at me as very detailed and considerate, “The discussions by the instructors were very inspiring because they were genuinely interested in us and dedicated to helping us see the philosophy of this curriculum. However my answer to the question of recommending this course is low because of the covid situation in our world. My answer is not because of the integrity of the course, the suggestions or the”gist" of the course. The instructors were excellent. The philosophy of equity and responsibility is also. But I cannot recommend this course in light of covid-19. I feel teaching EL through distance learning with integrity would not be possible." This response mirrors the few low scores I did see in the likeliness to recommend to a colleague or friend earlier, covid has made it much more difficult to make this style of course worthwhile.

On a positive note all the text relating to “not alone” is great feedback, with 6 people mentioning under “what went well in the session” that they felt they were not alone. Once again I am utterly confused by responses like 38 where special characters seem to have randomly replaced text, it is still legible but quite strange. There’s a lot to pull from actually reading comments here, so I’ll just mention one more - in responses 15, 18, 19, and 20 all mention the convention center setup as poor, and the chairs as uncomfortable, which is likely outside of anyone at Teaching Lab’s control but it seems like a non-ideal setting.

Word-topic Probabilities

The tidytext package provides a function to extract the per-topic-per-word probabilities called \(\beta\) from a LDA_VEM topic model with a number of topics proportional to the response types. Looking at the response probabilities per topic it looks like more time continues to be the biggest theme, and group work again emerges as a success story for best learning activities. “Nothing”, has the highest probability response to what could have improved your experience, which is a really good sign!

 

Return to Website

gatesdu@oregonstate.edu